A State-of-the-art Review on Multimodal Video Indexing

Efficient and effective handling of video documents depends on the availability of indexes. Manual indexing is unfeasible for large video collections. Effective indexing requires a multimodal approach in which either the most appropriate modality is selected or the different modalities are used in collaborative fashion. In this paper we focus on the similarities and differences between the modalities, and survey several methods aiming at automating the time and resource consuming process of video indexing. Furthermore, we put forward a unifying and multimodal framework, which views a video document from the perspective of its author. This framework forms the guiding principle for identifying index types, for which automatic methods are found in literature. It furthermore forms the basis for categorizing these different methods.